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Abstract.  Automated  orthorectification  of  raw  image  products  is  now  possible  based  on  the 
comprehensive  metadata  collected  by  Global  Positioning  Systems  and  Inertial  Measurement 
Unit  technology  aboard  aircraft  and  satellite  digital  imaging  systems,  and  based  on  emerging 
pattern-matching  and  automated  image-to-image  and  control  point  selection  capabilities  in 
many  advanced  image  processing  systems.  Automated  orthorectification  of  standard  aerial 
photography  is  also  possible  if  a  camera  calibration  report  and  sufficient  metadata  is 
available.  Orthorectification  of  historical  imagery,  for  which  only  limited  metadata  was 
available,  was  also  attempted  and  found  to  require  some  user  input,  creating  a  semi-automated 
process  that  still  has  significant  potential  to  reduce  processing  time  and  expense  for  the 
conversion  of  archival  historical  imagery  into  geospatially  enabled,  digital  formats, 
facilitating  preservation  and  utilization  of  a  vast  archive  of  historical  imagery.  Over  90 
percent  of  the  frames  of  historical  aerial  photos  used  in  this  experiment  were  successfully 
orthorectified  to  the  accuracy  of  the  USGS  100K  base  map  series  utilized  for  the  geospatial 
reference  of  the  archive.  The  accuracy  standard  for  the  100K  series  maps  is  approximately 
167  feet  (51  meters).  The  main  problems  associated  with  orthorectification  failure  were  cloud 
cover,  shadow  and  historical  landscape  change  which  confused  automated  image-to-image 
matching  processes.  Further  research  is  recommended  to  optimize  automated 

orthorectification  methods  and  enable  broad  operational  use,  especially  as  related  to  historical 
imagery  archives. 

Keywords,  orthorectification,  automated  orthorectification,  georegistration,  digital  metadata, 
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1  INTRODUCTION 

Remote  sensing  is  becoming  an  increasingly  important  science  for  advancing  the 
understanding  of  environmental  processes,  conditions,  changes,  and  for  monitoring  both 
human  and  ecological  health.  Key  societal  decision  making  processes  such  as  emergency 
response  and  land  use  planning  routinely  rely  on  information  derived  from  imagery  and 
remote  sensing  systems.  New  sensors,  systems  and  commercially  available  imagery  have 
changed  many  aspects  of  modern  life.  However,  even  with  the  significant  advancements  in 
sensor  technology  and  image  availability,  one  of  the  most  important  and  enduring  aspects  of 
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all  remotely  sensed  data  is  the  simple  fact  that  each  image  represents  a  sample  point  in  time 
that  is  preserved  for  countless  future  applications. 

Acquisition  of  aerial  photos  and  other  aircraft-based  imagery  has  been  regularly 
accomplished  since  the  1920s  and  represents  a  rich  source  of  historical  observation  data.  The 
value  of  historical  imagery  for  many  applications  has  been  documented  by  [1-8].  However, 
the  great  majority  of  this  data  still  exists  in  analog  form  only  (film  and/or  paper),  often  with 
minimal  metadata  and  cannot  be  easily  accessed  and/or  utilized  except  through  time- 
consuming  manual  search  and  indexing  methods.  Further,  to  be  utilized  in  modern 
Geographic  Information  Systems  (GIS),  analog-to-digital  conversion  and  spatial  geo- 
referencing  must  be  accomplished,  which  is  a  slow,  labor-intensive,  time  consuming  effort. 
One  of  the  great  data  management  challenges  for  the  geographic  community  will  be  the 
development  of  a  strategy  for  the  remedial  analog  to  digital  conversion  and  spatial  geo- 
referencing  of  the  historical  imagery  holdings  of  the  world. 

Orthorectification  is  a  three-dimensional  photogrammetric  correction  of  raw  imagery  that 
digitally  renders  the  image  into  a  cartographically  corrected  map-quality  image  tied  to  an 
earth  coordinate  system.  The  process  requires  knowledge  of  the  camera  system  (lens 
distortions,  focal  length),  the  collection  parameters  (flying  height,  percentage  of  overlap...) 
and  knowledge  of  the  imaged  terrain  such  as  real-world  coordinates  and  topographic 
elevation.  These  parameters  feed  complex  photogrammetric  equations  which  create 
mathematical  relationships  between  the  sensor,  the  image  and  the  target  surface.  Until 
recently,  most  of  these  parameters  were  manually  input  into  a  computer  model  and  processed 
by  skilled  technicians.  However,  with  the  advent  of  digital  imaging  systems,  on-board  Global 
Positioning  Systems  (GPS)  and  Inertial  Measurement  Unit  (IMU)  technology,  pattern 
recognition  technology,  digital  databases  of  previous  orthorectified  images  and  digital 
elevation  models  (DEM),  the  entire  orthorectification  process  can  now  be  automated  [9-11], 
resulting  in  tremendous  reduction  in  the  cost  and  time  for  processing  imagery. 

To  realize  the  full  breadth  of  remote  sensing  information  potential  relevant  to 
environmental  decision-making,  the  enormous  archives  of  analog  remote  sensing  data  must  be 
converted  to  digital  format  and  geo-referenced  into  a  geodetic  coordinate  system.  With  the 
emergence  of  automated  orthorectification  technology,  the  opportunity  exists  to  greatly 
increase  the  base  of  remotely  sensed  information  and  to  efficiently  serve  that  information,  via 
GIS/internet  services,  directly  to  the  decision  makers. 

This  paper  reports  on  a  pilot  research  project  to  apply  emerging  automated 
orthorectification  technology  to  historical  imagery  archives  and  evaluate  the  advantages  and 
disadvantages  that  this  technology  presents. 

2  BACKGROUND 

In  an  age  of  integrated  spatial  analysis,  imagery  is  most  useful  for  decision  makers,  when  it 
can  be  fully  integrated  with  other  data  sets  in  a  GIS  environment.  To  do  this,  a  process  of 
geo-registration  must  be  accomplished  where  the  image  is  tied  to  a  known  ground  coordinate 
system  and  re-sampled  into  map-quality  spatial  properties.  Orthorectification  is  the  same 
process  but  includes  elevation  data  to  further  refine  the  location  of  each  image  pixel.  The 
essential  problem  is  that  this  orthorectification  process  is  slow,  time  consuming  and  requires 
trained  remote  sensing  technicians  to  perform.  The  result  is  that  new  imagery  often  does  not 
get  into  a  usable  analytical  form  for  many  weeks  after  acquisition,  even  in  emergency 
situations.  This  also  means  that  much  of  our  historical  holdings  of  analog  aerial  photography 
will  never  be  converted  into  this  digital,  geospatial  format,  simply  because  the  process  is  too 
expensive. 

Although  small  compared  to  the  imagery  archives  of  other  agencies  like  the  United  States 
Department  of  Agriculture  (USDA)  and  the  U.S.  Geological  Survey  (USGS),  the 
Environmental  Protection  Agency  (EPA)  currently  maintains  a  Remote  Sensing  Archive 
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(RSA)  of  imagery  holdings  that  includes  over  5,000  rolls  and  75,000  individual  frames  of 
aerial  photos  and  associated  products  that  exist  primarily  in  analog  form  only.  For  these 
imagery  holdings  to  be  fully  effective  and  functional,  it  requires  the  massive  conversion  of  the 
RSA’s  analog  film  holdings  to  digital,  geo-referenced  form,  along  with  the  development  of 
appropriate  metadata.  Although  this  conversion  process  is  currently  underway,  it  is  a  time 
and  labor-intensive  process  that  will  require  at  least  ten  years  to  complete  at  current 
production  rates.  An  automated  ortho-rectification  process  promises  to  be  a  potentially  major 
breakthrough  in  the  overall  utilization  of  imagery  for  geospatial  analysis.  Imagery  that  once 
took  weeks  to  process  can  now  be  processed  in  minutes  for  emergency  and  time-critical 
responses.  Additionally,  we  will  have  a  practical  and  cost-efficient  method  for  converting 
historical  imagery  holdings  into  an  effective  geospatial  format. 

The  Automated  Imagery  Orthorectification  Pilot  (AIORP)  system  is  comprised  of  the 
best-of-breed  Commercial  Off  The  Shelf  (COTS)  products  that  are  compliant  with  Open 
Geospatial  Consortium  interface  specifications  and  are  integrated  through  platform- 
independent  code  that  enables  the  data  workflow  and  processing.  The  AIORP  is  designed  to 
provide  a  system  capable  of  fully  automated  orthorectification  of  full-frame  commercial 
satellite  images.  It  is  a  complete  end-to-end  system  where  no  user  interaction  is  required, 
except  for  the  submittal  of  the  raw  images.  The  control  data  required  for  automated 
orthorectification,  controlled  image  base  (CIB)  or  digital  ortho  quad  (DOQ)  and  digital  terrain 
elevation  data  (DTED,  DEM),  are  discovered  dynamically  and  are  automatically  used  by  the 
system  without  user  intervention.  The  architecture  is  scalable  and  allows  for  multiple 
orthorectification  processes  to  be  running  concurrently  in  a  distributed  fashion  across  multiple 
hardware  elements  made  available  to  the  system.  The  AIORP  system  is  based  on  COTS 
technology,  in  this  case  a  combination  of  the  Ionic  (Liege,  Belgium)  geospatial  catalog  and 
PCI  (Ontario,  Canada)  Geomatics  orthorectification.  Processing  is  done  on  a  dedicated 
hardware  cluster  that  is  capable  of  approximately  20  images  per  hour. 

Until  very  recently,  georeferencing  and  orthorectification  were  manual  processes  that  were 
accomplished  by  skilled  photogrammetric  technicians,  first  by  analog  instruments  that 
reprojected  the  corrected  image  into  cartographically  corrected  format,  and  then  later  by 
digital  photogrammetric  processing.  Under  the  best  circumstances,  the  entire 
orthorectification  process  was,  and  still  is,  technically  difficult,  time-consuming  and  labor- 
intensive.  This  is  the  primary  reason  that  most  of  the  historical  imagery  that  today  exists  in 
the  many  film  archives  of  the  government  and  private  industry,  has  NOT  been  converted  into 
a  geospatial,  GIS  friendly  format.  It  is  simply  too  expensive. 


2.1  The  Evolution  of  Automated  Orthorectification 

Although  not  fully  implemented,  automated  orthorectification  is  becoming  an  accepted  image 
processing  capability  in  the  world  of  remote  sensing.  Several  commercial  image  processing 
vendors  offer  standard  processing  packages  [12,  13]  for  an  automated  imagery 
orthorectification  process  and  related  applications  for  radar  and  other  types  of  imagery. 
Several  research  applications  have  shown  that  automated  orthorectification  of  satellite 
imagery  can  be  employed  as  a  standard  component  of  preliminary  image  processing  [14-17]. 
A  survey  of  image  registration  techniques  can  be  found  in  Brown  (1992)  and  Zitova  and 
Flusser  (2003)  [18,  19]  and  there  are  many  applications  of  automated  image  registration  or 
orthorectification  as  related  to  landscape  change  detection  [20].  Some  notable  examples  of 
automated  registration  or  orthorectification  techniques  include  the  following. 

Liu  and  Chen  (2009)  [21]  developed  a  methods  for  automated  orthorectification  of 
Formosat-2  imagery  using  a  four-step  process  that  includes;  1)  automatic  extraction  of 
Ground  Control  Regions,  (GCRs)  (2)  fast  image-to-image  matching,  3)  iterating  and  filtering 
of  GCRs,  and  4)  rigorous  orthorectification.  Results  showed  that  the  accurate  orthoimage 
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with  a  root  mean  square  error  of  less  than  1.5  pixels  could  be  automatically  generated  from 
one  standard  Formosat-2  in  55  minutes.  This  new  method  has  been  incorporated  into  the 
Formosat-2  automatic  image  processing  system  and  is  currently  being  used  to  produce 
orthoimages  on  a  daily-basis 

Leprince  et  al.  (2007)  [22],  achieved  an  automated  and  precise  orthorectification  of  SPOT 
imagery  by  developing  an  optimized  model  of  the  platform  geometry  that  is  able  to  achieve 
1/50  pixel  accuracy  for  SPOT  images  and  was  successfully  used  to  monitor  subtle  ground 
deformations  due  to  seismic  activity. 

The  National  Aeronautics  and  Space  Administration  (NASA)  recently  developed  an 
automated  orthorectification  software  system  for  use  with  Synthetic  Aperture  Radar  (SAR) 
data  that  allows  users  of  SAR  data  to  form  processed  images  and  other  high-level  products 
that  are  free  of  distortion  due  to  the  imaging  geometry  or  topography  changes  [23]. 


2.2  Automated  Image  Matching  and  Control  Point  Generation 

The  current  level  of  automation  in  image  rectification  is  made  possible  by  both  area  and 
feature-based  automated  image  matching  techniques  that  were  developed  by  several 
researchers  in  the  1990s  [24,  25].  In  the  area  based  techniques,  a  small  window  of  pixels  in  a 
target  image  is  compared  to  the  same  window  of  pixels  in  a  reference  image.  Correspondence 
or  correlation  measures  are  used  to  judge  the  goodness-of-fit  and  the  target  or  reference 
window  may  be  adjusted  to  determine  if  there  is  a  better  fit  which  is  generally  determined  by 
a  cross-correlation  procedure  [24].  Fast  Fourier  Transforms  are  used  for  each  window  with 
the  window  centers  used  as  control  point  for  the  transformation  parameters  [24,  26]. 

Feature-based  methods  utilize  control  points  based  on  contrast  thresholding  in  the  images, 
edge,  boundaries  and  other  identifiable  point  features  usually  derived  from  some  type  of 
image  segmentation  procedure.  Features  could  also  come  from  a  control-point  database  that 
may  have  been  developed  over  time  and  repeated  rectification  processing. 

Feature  and  control  point  correspondence  is  then  processed  and  the  image  resampled 
based  on  a  variety  of  potential  mathematical  techniques  including  linear  and  polynomial 
regression,  structural  programming  and  other  techniques  [24].  Area-based  techniques  work 
best  when  there  is  a  high  degree  of  similarly  in  the  target  and  reference  images  but  they  tend 
to  fall  apart  when  complex  warping  transformations  are  involved.  Feature -based  techniques 
are  more  robust  and  work  better  when  the  there  are  major  misalignment  issues  and  therefore 
complex  transformations  which  are  computationally  more  rigorous  [24]. 


3  THE  AIORP  EXPERIMENT 

This  project  was  Rinded  by  the  2006  Environmental  Protection  Agency’s  (EPA)  Advanced 
Monitoring  Initiative  (AMI)  [27]  Project  31,  Automated  Imagery  Orthorectification  Pilot 
(AIORP),  and  was  collaboration  between  the  EPA  and  U.S.  Intelligence  Community  (IC) 
conducted  jointly  by  EPA/IC  with  the  contract  assistance  of  SRA  Incorporated.  The  goal  of 
the  AIORP  was  to  develop  and  test  an  automated  imagery  geo-rectification  capability  that 
would  enable  raw  historical  imagery  products  to  be  quickly,  accurately  and  efficiently 
transformed  into  cartographically-corrected  and  geo-spatially  referenced  remote  sensing 
information  products  that  can  be  served  in  near-real  time  to  a  community  of  internal  and 
external  scientific  users.  This  system  utilizes  several  emerging  geospatial  and 
photogrammetric  software  capabilities  that  enable  automated  positioning,  retrieval  of  ortho 
and  elevation  information,  internal  and  external  control  point  establishment,  and  a  full 
orthorectification  and  mosaic  capability. 
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This  AIORP  system  is  patterned  after  an  IC  prototype  that  involved  strictly  commercial, 
off-the-shelf  components  and  software  [10].  By  significantly  increasing  the  digital  base  of 
remotely  sensed  information  and  by  making  it  easily  accessible  via  GIS/internet  services,  this 
research  could  provide  important  new  monitoring  capabilities  to  major  research  communities 
such  as  the  U.S.  Geological  Survey’s  (USGS)  Land  remote  Sensing  (LRS)  [28]  program,  the 
Global  Climate  Change  [29|  and  Global  Earth  Observation  (GEO)  [30]  communities  via  a 
practical  methodology  to  convert  and  preserve  vulnerable  and  irreplaceable  historical  remote 
sensing  information  and  facilitate  its  use  in  environmental  decision-making. 


3.1  The  Automated  Imagery  Orthorectification  Process. 

The  potential  for  automated  imagery  georegistration  and  orthorectification  has  been 
demonstrated  by  a  number  of  researchers  [31-33].  In  general,  two  emerging  technological 
capabilities  make  automated  orthorectification  possible.  First  is  the  digital  metadata  that  is 
recorded  and  delivered  with  the  imagery  based  on  the  satellite  sensor  or  digital  camera 
platform,  instrument  and  collection  parameters.  The  GPS/IMU  data  that  is  collected  often 
contains  all  of  the  critical  photogrammetric  data  necessary,  such  as  geographic  location, 
flying  height,  camera  lens  parameters,  tip,  tilt,  yaw  and  x,  y  and  z  coordinates  and  estimated 
errors.  Second  is  automated  image-to-image  matching  and  control  point  generation.  With 
this  data  the  imagery  is  basically  delivered  with  a  specific  math  model  for  the  sensors  or 
rational  polynomial  coefficients  (RPC)  which  fully  enable  the  orthorectification  process.  All 
that  is  then  needed  is  an  existing  orthoimage  from  a  standard  database,  such  as  the  Controlled 
Image  Base  (CIB))  or  Digital  Orthophoto  Quad  (DOQ)  databases,  an  elevation  model,  such  as 
the  USGS  National  Elevation  Dataset  (NED)  and  a  scripting  language  to  provide  I/O  and 
control  the  process  flow.  [34-36] 

From  a  conceptual  standpoint,  the  AIORP  process  basically  works  in  the  following  way: 

1.  An  image  is  automatically  ingested  into  the  processing  node. 

2.  Interior  orientation  parameters  (a  sensor  specific  math  model,  or  focal  length, 
principal  point  of  symmetry,  radial  lens  distortion,  flight  height,  fiducial 
marks,  etc)  are  read  from  a  file  or  from  digital  metadata  or  header  file  that 
accompanies  the  image. 

3.  The  basic  geographic  footprint  is  calculated  for  the  image. 

4.  A  digital  elevation  model  and  reference  ortho-image  are  extracted  from  existing 
databases  covering  the  geographic  footprint  of  the  image. 

For  this  project,  the  digital  elevation  models  were  extracted  from  the  U.S.  Geological 
Survey’s  NED  database.  The  NED  is  the  primary  elevation  data  product  of  the 
USGS  and  is  a  seamless  dataset  with  the  best  available  raster  elevation  data  of  the 
conterminous  United  States,  Alaska,  Hawaii,  and  territorial  islands.  The  NED  has  a 
resolution  of  one  arc-second  (approximately  30  meters)  for  the  conterminous  United 
States  and  an  absolute  spatial  accuracy  of  7  meters. 

The  geographic  footprint  of  the  target  image  is  then  used  to  retrieve  a  geometrically 
corrected  image  from  a  seamless  database  of  existing  orthorectified  imagery.  For  this 
experiment  two  sets  of  existing  orthoimagery  were  utilized;  the  National  Geospatial 
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Intelligence  Agency  (NIMA)  Controlled  Image  base  (CIB)  and  the  U.  S.  Geological 
Survey’s  Digital  Orthophoto  Quad  (DOQ)  program  [35,  36] 

The  Controlled  Image  Base  (CIB)  is  a  dataset  of  orthophotos,  made  from  rectified 
grayscale  aerial  images.  CIB  supports  various  military  and  intelligence  geospatial 
information  requirements.  CIB  data  are  produced  from  digital  source  images  that  can 
be  converted  to  meet  the  requirements  of  the  CIB  specification  at  one  of  the  registered 
resolutions  defined  in  MIL-STD-241 1-1.  The  accuracy  of  CIB  products  can  vary  but 
for  this  application  it  was  5  meters  [35]. 

A  USGS  digital  orthophoto  quadrangle  (DOQ)  is  a  computer-generated  image  of  an 
aerial  photograph  in  which  image  displacement  caused  by  terrain  relief  and  camera 
tilts  has  been  removed.  It  combines  the  image  characteristics  of  a  photograph  with  the 
geometric  qualities  of  a  map.  Imagery  resolution  is  one  meters  and  the  geodetic 
spatial  accuracy  is  7  meters  root-mean  square-error  [36]. 

5.  Image-to-image  matching  is  accomplished  based  on  the  pattern  recognition,  and 
control  points  are  generated,  or  read  from  a  file. 

6.  Exterior  orientation  parameters  are  calculated. 

7.  Rigorous  orthorectification  is  performed. 

8.  After  a  basic  quality  control  screening,  the  new  ortho  image  is  delivered  to  a 
master  database  or  published  to  a  web  service. 


3.2  Hardware  Configuration 

The  actual  hardware  and  software  configuration  of  the  AIORP  process  was  altered  from  the 
original  prototype  because  the  project  focused  primarily  on  the  question  of  orthorectification 
and  not  on  overall  end-user  based  distribution  and  utilization.  However,  the  configuration  is 
reported  here  for  sake  of  clarity.  The  prototype  system,  as  reported  by  Smith  et  al.  [10]  and 
shown  in  Figure  1,  consists  of  a  4  terabyte  (TB)  RAID  array  for  storage,  a  front-end  server  for 
storage  and  web  distribution,  an  applications  server  for  geospatial  and  photogrammetric 
control  and  a  database  server,  all  on  a  1  gigabit  (gb)  ethernet  with  eight  blade  processing 
workstations  for  photogrammetric  processing  and  orthorectification  production.  The  actual 
configuration  in  this  experiment  included  a  front  end  server,  a  4  TB  RAID  and  three  blade 
processing  workstations. 
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3.3  Software  Configuration 

The  software  capabilities  to  perform  this  type  of  workflow  processing  are  now  becoming 
more  widely  available  in  the  remote  sensing  community.  For  this  study  PCI  Geomatica 
(Richmond  Hill,  Ontario,  Canada)  was  utilized  with  shell  scripts  used  to  control  the  workflow 
process.  Open  source  versions  of  orthorectification  are  available  in  packages  such 
as  Orthomap  : 

(http://www.remotesensingtools.com/20Q7/12/10/auto-orthorectification-tool-orthomap/) 

and,  OSSIM 

(http://ossim.telascience.org/ossimdata/Documentation/OSSIM  GUI  howto.doc). 

4  THE  CRITCAL  ROLE  OF  DIGITAL  METADATA 


With  the  advent  of  modem  geospatial  technologies,  such  as  Global  Positioning  Systems 
(GPS)  and  Inertial  Measurement  Unit  (IMU)  capabilities,  remote  sensing  satellites  and 
aircraft  can  capture  critical  photogrammetric  parameters,  such  as  nadir  position,  flight  height, 


Ortho  1 

Front-End  Server 
Integrity  Dual  AMD  Opteron 
(1.8GHz  W/3G  RAM) 


1  OBEKiernet 


{ 


Foundry  1 GB  Switch 


1  OB  Ethernet 


Ortho  2 

Application  Server 
Dell  Power  Edge  6600 
(Dual  Xeon  2.2  GHz  w/  4GB  RAM) 


Ortho  3 

Data  Base  Server 
Dell  Power  Edge  6600 
(Quad  Xeon  2.2  GHzw/  4GB  RAM) 


4  TB  HDD  RAID 


Compute  Nodes 

Dual  AMD  Opteron 
(1.8GHz  W/3G  RAM) 
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Fig.  1.  The  original  auto  orthorectification  processing  cluster  [10]. 
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tip,  tilt,  pitch,  yaw.. .etc,  with  considerable  precision  and  as  an  integral  part  of  the  image  data 
header  fde  or  the  associated  metadata  file.  This  greatly  facilitates  the  basic  mathematical 
resection  that  is  the  fundamental  math  process  in  orthorectification.  However,  when  dealing 
with  historical  imagery,  this  type  of  metadata  is  not  likely  to  be  available  and  two  special 
cases  must  be  considered  for  ortho-image  processing.  First,  a  digital  metadata  file  can  be 
created  from  critical  camera,  sensor  and  platform  parameters  and  existing  analog  data  such  as 
camera  calibration  reports  and  flights  logs,  and  second,  this  information  cannot  be  developed 
from  existing  analog  metadata  and  a  generic  metadata  file  must  be  created  from  a  limited 
information  such  as  fiducial  locations,  scale  and  flight  height,  all  which  can  be  measured  from 
the  image.  In  either  case  the  overall  process  must  require  some  user  input  and  becomes  semi- 
automated. 

Although  not  as  attractive  as  a  fully  automated  system,  the  semi-automated  process  still 
represents  tremendous  efficiencies  over  manual  methods.  Since  most  historical  aerial 
photography  is  collected  and  stored  on  rolls  of  film,  usually  taken  with  the  same  camera  over 
the  same  basic  area,  even  a  semi-automated  orthorectification  capability  would  permit  an 
analyst  to  set  up  one  metadata  file  for  the  entire  roll  and  run  the  automated  orthorectification 
process  on  several  hundred  frames  of  imagery  with  minimal  user  input.  Manual 
orthorectification  methods  often  take  upwards  of  1  hour  per  frame  so  that  the  efficiencies  to 
be  realized  are  still  very  significant. 


5  THE  PROCESSING  TRIALS 

Several  iterative  trials  and  several  individual  automated  ortho-image  processing  tests  were 
conducted  during  2006.  Also,  the  previous  prototype  results  are  reported  here  for  reference. 
The  results  of  both  trials  are  summarized  in  Table  1.  All  imagery  was  1 1-bit  panchromatic  or 
black  and  white.  The  satellite  images  averaged  about  13,000  x  13,000  pixels  at  one-meter 
ground  resolution.  The  aerial  photos  averaged  about  14,000  x  14,000  pixels  at  variable  ground 
resolution  but  usually  sub-meter.  The  initial  test  involved  satellite  imagery  with  digital 
metadata  and  was  very  efficient  except  where  the  metadata  was  insufficient  (i.e.,  corrupted  or 
incomplete).  This  was  a  test  of  a  fully  automated  system  and  no  attempts  were  made  to 
develop  a  solution  manually  or  in  a  semi-automated  manner.  Results  show  that  full  satellite 
scenes  could  be  completely  orthorectified  to  CIB  standards  in  an  average  of  less  than  15 
minutes  per  scene.  The  EPA  tests  in  Waynesboro,  Virginia  and  Leviathan  Mine,  California 
were  small  tests  designed  around  specific  potential  problems  [37].  In  Waynesboro,  historical 
imagery  dating  back  to  1937  was  utilized  to  determine  if  basic  landscape  change  would  affect 
the  automated  control  point  and  image-to-image  registration  processes.  In  fact,  some  of  the 
1937  imagery  failed  the  orthorectification  process  because  of  significant  level  of  landscape 
change  that  made  automated  image  matching  and  control  point  matching  very  difficult. 
Figure  2  shows  the  1937  and  1994  images  of  a  portion  of  the  area,  demonstrating  the  level  of 
landscape  change  involved  [37]. 

In  the  Leviathan  test,  extreme  elevation  change  proved  to  extend  the  processing  time 
significantly,  as  might  be  expected,  but  low  contrast  and  poor  quality  in  the  reference  DOQ 
data  appeared  to  be  the  biggest  problem.  In  the  EPA-Ohio  test,  relatively  smooth  and 
efficient  semi-automated  orthorectification  was  achieved  with  high  quality  imagery  over  a 
relatively  flat  terrain.  The  average  processing  time,  once  the  manual  set-up  was 
accomplished,  was  only  2.25  minutes  per  frame  [37].. 

However,  the  major  test  was  conducted  during  August  and  September  of  2006  at  the  EPA 
Remote  Sensing  Archive  in  Las  Vegas,  Nevada.  Random  rolls  of  archival  historical  photos 
were  selected  and  scanned  at  approximately  1800  dpi  (approximately  a  15  micron  scan). 
DOQ  and  NED  data  were  collected  for  reference  and  processing  purposes.  Semi-automated 
set-up  methods  were  used  which  involved  the  editing  of  basic  camera  and  photogrammetric 


Journal  of  Applied  Remote  Sensing,  Vol.  3,  033552  (2009) 


Page  8 


Downloaded  from  SPIE  Digital  Library  on  05  Jan  2010  to  131.84.11.215.  Terms  of  Use:  http://spiedl.org/terms 


parameters  and  the  manual  selection  of  a  minimum  (3-7)  number  of  control  points  per  photo 
from  the  reference  ortho  [37]. 

The  results  of  all  the  automated  orthorectification  experiments  are  listed  in  Table  1.  In 
general,  the  semi-automated  process  provided  acceptable  results  with  395  out  of  436  images 
being  orthorectified  to  some  level  of  accuracy.  Although  many  did  not  meet  the  accuracy 
standard  of  the  USGS  DOQ  program,  they  still  are  acceptable  for  basic  database  storage  and 
for  geospatial  imagery  archive  purposes.  Also,  additional  research  and  development  will 
likely  lead  to  improvements  in  the  spatial  accuracy  of  the  orthorectification  process. 

However,  the  major  test  was  conducted  during  August  and  September  of  2006  at  the  EPA 
Remote  Sensing  Archive  in  Las  Vegas,  Nevada.  Random  rolls  of  archival  historical  photos 
were  selected  and  scanned  at  approximately  1800  dpi.  DOQ  and  National  Elevation  Data 
were  collected  for  reference  and  processing  purposes.  Semi-automated  set-up  methods  were 
used  which  involved  the  editing  of  basic  camera  and  photogrammetric  parameters  and  the 
manual  selection  of  a  minimum  number  of  control  points. 


Table  1.  Summary  of  Auto-Orthoreetification  Results. 


Test 

Name 

Platform 

Type/ 

Sensor 

Image 

Type 

Avg 

Image 

Size 

Digit 

Total 

Images 

Completed/ 

Failed 

Orthos 

Average 

Time/Image 

(mm:ss) 

RMSE 

(meters) 

Reason  for 
Failures 

IC-l 

Satellite/ 

(Ikonos) 

Pan 

328  mb 

11  bit 

31 

22/9 

14:55 

<  5  m 

corrupted 

metadata 

EPA 

Virginia 

Aircraft/ 

camera 

B&W 

206  mb 

11  bit 

10 

8/2 

3:25 

<  10  m 

landscape 

change 

EPA 

Ohio 

Aircraft/ 

camera 

B&W 

212  mb 

11  bit 

61 

61/0 

2:25 

<  10  m 

N/A 

EPA 

Leviathan 

Aircraft/ 

camera 

B&W 

208  mb 

11  bit 

5 

5/0 

48:00 

<  15  m 

N/A 

EPA 

Aircraft/ 

B&W 

224  mb 

11  bit 

436 

395/41 

31:35 

<45  m 

Low  contrast 

Nevada  camera  Open  water 

Geometery 
Cloud  Cover 
Insufficient 
Metadata 
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Fig.  2.  A  1937  image  (top)  and  a  1994  image  (bottom)  of  the  area  around  Waynesboro,  Virginia 
demonstrating  the  level  of  landscape  change  that  can  cause  problems  with  automated  image-to- 
image  matching  and  control  point  identification  that  are  a  fundamental  process  in  automated  or 
semi-automated  image  orthorectification. 
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Of  the  images  that  failed  during  the  orthorectification  process,  the  primary  reasons  for 
failure  include: 

Incorrect  or  insufficient  metadata, 

Panoramic  or  oblique  camera  orientations, 

Too  much  landscape  change  between  the  historical  image  and  the  reference 
image, 

Open  water,  shadows  or  cloud  cover  in  the  raw  image, 

Poor  tonal  quality  or  low  contrast  in  the  reference  DOQQ  image. 


7  DISCUSSION 

Although  a  limited  trial,  this  project  represents  an  important  initial  step  in  the  development  of 
an  automated  and/or  semi-automated  orthorectification  process  that  can  be  functional  even  for 
historical  imagery  for  which  little  or  no  metadata  exists.  An  initial  metadata  file  can  be 
created  from  the  analog  information  such  as  flight  logs  and  camera  calibration  reports.  Even 
if  this  analog  data  can  not  be  recovered,  a  semi-automated  orthorectification  process  can  still 
be  accomplished  by  estimating  the  collection  parameters  and  manually  selecting  a  small 
number  of  control  points  between  the  database  ortho  image  and  the  raw  image.  In  terms  of 
processing  efficiency,  this  is  still  a  significant  improvement  of  purely  manual  methods  and 
could  open  the  door  to  the  orthorectification  of  a  huge  global  database  of  historical  aircraft 
and  satellite  imagery.  In  this  experiment,  single  frames  were  processed  independently  so  that 
issues  related  to  individual  image  characteristics  and/or  geometries  could  be  easily  identified. 
However,  in  typical  photogrammetric  mapping  operations  an  aero-triangulation  solution  and 
bundle  adjustment  could  be  easily  be  performed  and  would  be  likely  to  improve  results. 

The  primary  problems  are  a  lack  of  accurate  metadata,  including  camera  calibration 
reports  and  landscape  change  over  time  which  complicates  image  registration  and  automated 
control  point  generation.  These  could  be  potentially  overcome  by  gaining  access  to  the 
archive  of  camera  reports,  especially  in  a  digital  format.  Further,  landscape  change  could  be 
minimized  by  creating  historical  databases  of  orthorectified  imagery  to  be  used  as  the  base 
map. 


8  CONCLUSIONS  AND  RECOMMENDATIONS 
8.1  Findings 

Significant  findings  of  this  research  include  both  the  relative  ease  with  which  automated 
orthorectification  can  be  accomplished  when  sufficient  metadata  and  control  information 
exist,  and,  the  fact  that  historical  imagery,  for  which  little  or  no  metadata  exists,  can  still  be 
orthorectified  with  a  semi-automated  process,  which  is  still  significantly  more  efficient  than 
traditional  methods.  Further  research  and  development  in  automated  and  semi-automated 
orthorectification  is  clearly  justified  and  strongly  encouraged.  Logical  development  of 
automated  or  semi-automated  orthorectification  processing  might  include  the  following  tasks. 
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8.2  Camera  Calibration  Database 


The  creation  of  a  comprehensive  camera  calibration  database  would  be  extremely  beneficial 
to  model  setup  /  metadata  problems.  The  USGS  Optical  Science  Laboratory  has  been 
providing  camera  calibration  services  for  decades  and  maintains  a  rich  data  base  of  camera 
calibration  reports  [38].  Although  most  of  these  currently  exist  in  analog  form,  it  would  be 
clearly  beneficial  to  the  entire  mapping  community  to  have  these  all  available  in  a  standard 
xml  format  so  that  they  could  be  ingested  by  any  image  processing  system.  The  creation  of  a 
database  that  could  feed  an  automated  orthorectification  process  would  represent  a  major 
improvement  in  process  efficiency  and  accuracy.  The  USGS  Optical  Science  Laboratory 
should  be  encouraged  to  create  such  a  digital  product 

8.3  Further  Development  of  the  Orthorectification  Process 

It  is  clear  that  some  of  the  problems  with  the  historical  imagery  rectification  could  be 
improved  by  automating  and  including  several  standards  mapping  methods  such  as 
performing  an  aero  triangulation  solution  and  bundle  adjustment.  Also,  the  development  of  a 
generic  air  photo  camera  model  could  improve  the  overall  process  flow. 

8.4  Historical  Ortho  and  DEM  Databases 

The  problem  with  landscape  change  confusing  automated  registration  could  be  solved  by 
creating  temporal  orthoimage  databases,  especially  in  urban  areas  where  landscape  change  is 
likely  to  be  more  pronounced.  Further,  this  concept  could  also  be  extended  to  historical  DEM 
databases,  especially  in  areas  of  major  topographic  change. 


8.5  Second  Generation  Orthophoto  Processing 

There  are  some  natural  parallels  here  with  an  emerging  concept  of  second-generation 
orthophoto  processing.  Second  Generation  Orthophoto  processing  is  a  term  that  refers  to  the 
production  of  an  ortho  photo  that  is  produced  from  new  photography  and  2nd  generation 
ground  control  points  derived  from  existing  maps  or  ortho  photos  and  existing  elevation 
model  data  [39].  There  are  significant  resource  efficiencies  to  be  realized  from  this  approach 
since  it  is  less  expensive  and  faster  to  produce.  However,  the  accuracy  budget  must  allow  for 
new  orthoimages  with  degraded  accuracy.  In  general,  this  concept  applies  to  large  scale 
mapping  projects  with  engineering-level  map  accuracy  standards.  The  conversion  of 
historical  imagery  archives  is  less  concerned  with  map  accuracy  standard  than  it  is  with  the 
basic  conversion  of  historical  imagery  to  a  digital  format  with  basic  and  fundamental 
geographic  correction  and  reference.  However,  experiences  with  the  second  generation 
orthophoto  process  might  provide  useful  insight  into  automated  orthophoto  production  and 
should  be  explored  in  any  future  research  effort. 

8.6  The  Concept  of  Volumetric  Appearance  Modeling 

An  emerging  alternative  technology  to  standard  photogrammetry  is  that  of  Volumetric 
Appearance  Modeling  (VAM).  The  VAM  concept  is  based  on  the  3-D  data  model  of  a  voxel 
instead  of  the  standard  2-D  pixel.  The  voxel  is  a  cubical  volume  element  with  both  a 
Gaussian  occupancy  probability  and  a  Bayesian  distribution  probability  [40-41].  The  voxel 
concept  is  primarily  concerned  with  change  detection,  especially  in  the  third  dimension  and  is 
based  on  image  texture  and  variability  from  multiple  images  from  multiple  look  angles,  as 
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might  be  expected  from  wide  area  video  surveillance  [40].  However,  it  also  represents  an 
alternative  approach  to  standard  photogrammetric  orthorectification.  An  automated  sensor 
model  geographic  correction,  based  on  tying  voxel  parameters  to  3-d  control  points  has  shown 
promise  in  satellite  images,  achieving  sub-pixel  levels  of  registration  accuracy  [41].  Since 
the  voxel  already  represents  a  3 -dimensional  data  structure,  some  of  the  problems  with 
photogrammetric  elevation  correction  are  inherently  minimized.  Although  the  concept  is 
based  on  existing  VAM  data  from  multiple  images  and  additional  research  will  be  required,  it 
could  potentially  offer  an  interesting  alternative  to  standard  orthorectification  problems. 

This  paper  has  discussed  a  limited  but  successful  attempt  at  automated  and  semi- 
automated  orthorectification  which  could  have  significant  implications  in  future  utilization  of 
historical  images  of  all  types.  The  ability  to  store,  distribute,  and  serve  historical  imagery  in 
geospatial  formats  from  multiple  sources  could  have  significant  implications  for  a  variety  of 
geographic  research  areas  such  as  land  use  and  land  cover  monitoring,  global  change  and 
landscape  ecology. 
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